Demo 2: Feature Selection, Feature Importance and Model Interpretation

Cd13652 C6 L4 Demo 2 V3

Understanding Feature Selection, Importance, and Interpretability in Machine Learning

Feature selection, importance, and model interpretability are critical for building efficient predictive models.

Data Preparation

Dataset: Modified CNN pred data, cleaned by removing columns with high missing values.
Target: Classified next-day returns direction (up/down) of the S&P 500 index.
Features: 70 indicators including technical indicators, stock components, treasury yields.

Methods Used

Random Forest Classifier: Suitable for small datasets, non-sensitive to multicolinearity.
Feature Significance: Evaluations using permutation importance to understand feature performance impact.

Key Processes

Examine correlation matrices to eliminate highly correlated features.
Split datasets into training and testing subsets (70% train, 30% test).
Train random forest model; assess accuracy (test score ~93%).

Insights

S&P 500 features significantly influence model performance.
Utilize permutation importance to measure features' impact; visualize importance scores for clarity.
Apply methods like Recursive Feature Elimination (RFE) to refine feature selection.

Recommendations

Consider additional feature selection approaches in scikit-learn.
Validate feature importance to ensure reasonable alignment with correlation data.